A Maximum Entropy Method for Language Modelling
نویسندگان
چکیده
The language models used for automatic speech recognition (ASR) are often based on very simple Markov models. This paper presents an overview of a more powerful modelling technique, Maximum Entropy (ME), and its application in langauge modelling. Preliminary results indicate that ME models are viable for this task, and perform slightly better than the traditional models.
منابع مشابه
Distant bigram language modelling using maximum entropy
In this paper, we apply the maximum entropy approach to so-called distant bigram language modelling. In addition to the usual unigram and bigram dependencies, we use distant bigram dependencies, where the immediate predecessor word of the word position under consideration is skipped. The contributions of this paper are: (1) We analyze the computational complexity of the resulting training algor...
متن کاملAdaptive Statistical Language Modelling
The trigram statistical language model is remarkably successful when used in such applications as speech recognition. However, the trigram model is static in that it only considers the previous two words when making a prediction about a future word. The work presented here attempts to improve upon the trigram model by considering additional contextual and longer distance information. This is fr...
متن کاملCollocation Lattices and Maximum Entropy Models
Maximum entropy framework proved to be expressive and powerful for the statistical language modelling, but it suffers from the computational expensiveness of the model building. The iterative scaling algorithm that is used for the parameter estimation is computationally expensive while the feature selection process requires to estimate parameters of the model for many candidate features many ti...
متن کاملFeature Lattices for Maximum Entropy Modelling
Maximum entropy framework proved to be expressive and powerful for the statistical language modelling, but it suffers from the computational expensiveness of the model building. The iterative scaling algorithm that is used for the parameter estimation is computationally expensive while the feature selection process might require to estimate parameters for many candidate features many times. In ...
متن کامل{32 () Feature Lattices and Maximum Entropy Models Ref:mach1379-rm Editor: Ray Mooney
The maximum entropy framework has proved to be expressive and powerful for statistical language modelling, but it suuers from the computational expensiveness of model building. The iterative scaling algorithm that is used for parameter estimation is rather slow while the feature selection process might require parameters for many candidate features to be estimated many times. In this paper we p...
متن کامل